Systems and methods for anatomic motion compensation

ABSTRACT

A method includes receiving a first set of image data representing passageways at a first cyclical motion state, receiving a second set of image data representing the passageways at a second cyclical motion state, receiving pose data for points describing a shape of an instrument, and comparing the shape of the instrument to the first and second sets of image data. The comparing includes assigning match scores to the sets of image data by comparing each to the shape of the instrument and determining a selected set of image data that matches the shape. The method further includes identifying a phase of a cyclical anatomical motion, generating a command signal indicating an intended movement of the instrument, adjusting the command signal to include an instruction for a cyclical instrument motion of the instrument based on the phase of the cyclical anatomical motion, and causing the intended movement of the instrument.

RELATED APPLICATIONS

This patent application claims priority to and the benefit of the filingdate of U.S. Provisional Patent Applications 61/969,510, titled “Systemsand Methods for Anatomic Motion Compensation to Register InterventionalInstruments,” filed Mar. 24, 2014, and U.S. Provisional PatentApplication 62/052,802, titled “Systems and Methods for Anatomic MotionCompensation,” filed Sep. 19, 2014, which are all incorporated byreference herein in their entirety

FIELD

The present disclosure is directed to systems and methods for navigatinga patient anatomy to conduct a minimally invasive procedure, and moreparticularly to systems and methods for dynamically deforming ananatomical passageway model for display.

BACKGROUND

Minimally invasive medical techniques are intended to reduce the amountof tissue that is damaged during interventional procedures, therebyreducing patient recovery time, discomfort, and harmful side effects.Such minimally invasive techniques may be performed through naturalorifices in a patient anatomy or through one or more surgical incisions.Through these natural orifices or incisions clinicians may insertinterventional instruments (including surgical, diagnostic, therapeutic,or biopsy instruments) to reach a target tissue location. To reach thetarget tissue location, a minimally invasive interventional instrumentmay navigate natural or surgically created passageways in anatomicalsystems such as the lungs, the colon, the intestines, the kidneys, theheart, the circulatory system, or the like. Teleoperated interventionalsystems may be used to insert and position the interventional instrumentwithin the patient anatomy. During navigation of the interventionalinstrument, the clinician may be assisted by receiving preoperative orintraoperative images of the patient anatomy registered with theposition of the interventional instrument. Many regions of the patientanatomy are dynamic in normal function (e.g., heart, lungs, kidneys,liver, blood vessels). Conventional methods of registering aninterventional instrument with images of a dynamic anatomy areinadequate in some ways. For example, conventional methods ofregistration using electromagnetic (EM) sensors attempt to time sensingmeasurements at baseline conditions (e.g. full exhalation, fullinhalation). However, taking such measurements precisely insynchronization with the baseline condition leads to inconsistenciesbetween perceived and actual instrument positions. Furthermore, thediscontinuous nature of the baseline-only measurements may provide ajerky, non-intuitive experience for the clinician viewing displayedimages of the registration. Systems and methods are needed for improvedregistration of an interventional instrument with images of a dynamicanatomy.

SUMMARY

The embodiments of the invention are summarized by the claims thatfollow the description.

In one embodiment, a method of modeling a cyclic anatomical motioncomprises receiving a pose dataset for an identified point on aninterventional instrument retained within and in compliant movement witha cyclically moving patient anatomy for a plurality of time parameters.The method also includes determining a set of pose differentials for theidentified point with respect to a reference point at each of theplurality of time parameters and identifying a periodic signal for thecyclic anatomical motion from the set of pose differentials.

In another embodiment, a method of tracking an interventional instrumentwithin a plurality of passageways of a patient anatomy during cyclicalanatomical motion comprises receiving a set of image data representingthe plurality of passageways at a first cyclical motion state andreceiving a set of image data representing the plurality of passagewaysat a second cyclical motion state. The method also includes receivingpose data for a plurality of points describing a shape of aninterventional instrument positioned with the plurality of passagewaysat a first time parameter and comparing the shape of the interventionalinstrument at the first time parameter to the sets of image datarepresenting the plurality of passageways at the first and secondcyclical motion states. The method further includes assigning a matchscore to each of the sets of image data and determining a selected setof image data for the best match score.

In another embodiment, a method of tracking a cyclic anatomical motioncomprises receiving a pose dataset for an identified point on aninterventional instrument retained within and in compliant movement witha cyclically moving patient anatomy for a plurality of time parametersand identifying a periodic signal for the cyclic anatomical motion. Thecyclical anatomical motion is respiration.

Additional aspects, features, and advantages of the present disclosurewill become apparent from the following detailed description.

BRIEF DESCRIPTIONS OF THE DRAWINGS

Aspects of the present disclosure are best understood from the followingdetailed description when read with the accompanying figures. It isemphasized that, in accordance with the standard practice in theindustry, various features are not drawn to scale. In fact, thedimensions of the various features may be arbitrarily increased orreduced for clarity of discussion. In addition, the present disclosuremay repeat reference numerals and/or letters in the various examples.This repetition is for the purpose of simplicity and clarity and doesnot in itself dictate a relationship between the various embodimentsand/or configurations discussed.

FIG. 1 is a telerobotic interventional system, in accordance with anembodiment of the present disclosure.

FIG. 2 illustrates an interventional instrument system utilizing aspectsof the present disclosure.

FIG. 3a illustrates a method for modeling cyclic anatomical motionaccording to an embodiment of this disclosure.

FIG. 3b illustrates a method for modeling cyclic anatomical motionaccording to another embodiment of this disclosure.

FIG. 4 illustrates a method of tracking an interventional instrumentwithin a patient anatomy during cyclic anatomical motion.

FIG. 5 illustrates maximum and minimum state pose differentials for apoint P of an interventional instrument during cyclic anatomical motion.

FIG. 6 illustrates an intervention instrument pose dataset compared tothe anatomical image data at maximum and minimum states according to anembodiment of the present disclosure.

FIGS. 7 and 8 illustrate look-up tables according to differentembodiments of the present disclosure.

FIG. 9 illustrates a method of using periodic signal data related to ananatomical motion cycle.

FIG. 10 illustrates a method of modifying preoperative anatomical imagedata based on intraoperative image data.

DETAILED DESCRIPTION

In the following detailed description of the aspects of the invention,numerous specific details are set forth in order to provide a thoroughunderstanding of the disclosed embodiments. However, it will be obviousto one skilled in the art that the embodiments of this disclosure may bepracticed without these specific details. In other instances well knownmethods, procedures, components, and circuits have not been described indetail so as not to unnecessarily obscure aspects of the embodiments ofthe invention. And, to avoid needless descriptive repetition, one ormore components or actions described in accordance with one illustrativeembodiment can be used or omitted as applicable from other illustrativeembodiments.

The embodiments below will describe various instruments and portions ofinstruments in terms of their state in three-dimensional space. As usedherein, the term “position” refers to the location of an object or aportion of an object in a three-dimensional space (e.g., three degreesof translational freedom along Cartesian X, Y, Z coordinates). As usedherein, the term “orientation” refers to the rotational placement of anobject or a portion of an object (three degrees of rotationalfreedom—e.g., roll, pitch, and yaw). As used herein, the term “pose”refers to the position of an object or a portion of an object in atleast one degree of translational freedom and to the orientation of thatobject or portion of the object in at least one degree of rotationalfreedom (up to six total degrees of freedom). As used herein, the term“shape” refers to a set of poses, positions, or orientations measuredalong an elongated object.

Referring to FIG. 1 of the drawings, a telerobotic interventional systemfor use in, for example, surgical, diagnostic, therapeutic, or biopsyprocedures, is generally indicated by the reference numeral 100. As willbe described, the telerobotic interventional systems of this disclosureare generally under the teleoperational control of a surgeon. However,for some procedures or sub-procedures, the telerobotic interventionalsystem may be under the partial or full control of a computer programmedto perform the procedure or sub-procedure. As shown in FIG. 1, thetelerobotic interventional system 100 generally includes a roboticassembly 102 mounted to or near an operating table O on which a patientP is positioned. An interventional instrument system 104 is operablycoupled to the robotic assembly 102. An operator input system 106 allowsa surgeon or other type of clinician S to view the surgical site and tocontrol the operation of the interventional instrument system 104.

The operator input system 106 may be located at a surgeon's consolewhich is usually located in the same room as operating table O. However,it should be understood that the surgeon S can be located in a differentroom or a completely different building from the patient P. Operatorinput system 106 generally includes one or more control device(s) forcontrolling the interventional instrument system 104. The controldevice(s) may include any number of a variety of input devices, such ashand grips, joysticks, trackballs, data gloves, trigger-guns,hand-operated controllers, voice recognition devices, touch screens,body motion or presence sensors, or the like. In some embodiments, thecontrol device(s) will be provided with the same degrees of freedom asthe interventional instruments of the robotic assembly to provide thesurgeon with telepresence, or the perception that the control device(s)are integral with the instruments so that the surgeon has a strong senseof directly controlling instruments. In other embodiments, the controldevice(s) may have more or fewer degrees of freedom than the associatedinterventional instruments and still provide the surgeon withtelepresence. In some embodiments, the control device(s) are manualinput devices which move with six degrees of freedom, and which may alsoinclude an actuatable handle for actuating instruments (for example, forclosing grasping jaws, applying an electrical potential to an electrode,delivering a medicinal treatment, or the like).

The robotic assembly 102 supports the interventional instrument system104 and may comprise a kinematic structure of one or more non-servocontrolled links (e.g., one or more links that may be manuallypositioned and locked in place, generally referred to as a set-upstructure) and a robotic manipulator. The robotic assembly 102 includesplurality of actuators (e.g., motors) that drive inputs on theinterventional instrument 104. These motors actively move in response tocommands from the control system (e.g., control system 112). The motorsinclude drive systems which when coupled to the interventionalinstrument 104 may advance the interventional instrument into anaturally or surgically created anatomical orifice and/or may move thedistal end of the interventional instrument in multiple degrees offreedom, which may include three degrees of linear motion (e.g., linearmotion along the X, Y, Z Cartesian axes) and three degrees of rotationalmotion (e.g., rotation about the X, Y, Z Cartesian axes). Additionally,the motors can be used to actuate an articulable end effector of theinstrument for grasping tissue in the jaws of a biopsy device or thelike.

The robotic interventional system 100 also includes a sensor system 108with one or more sub-systems for receiving information about theinstruments of the robotic assembly. Such sub-systems may include aposition sensor system (e.g., an electromagnetic (EM) sensor system); ashape sensor system for determining the position, orientation, speed,pose, and/or shape of the catheter tip and/or of one or more segmentsalong a flexible body of instrument 104; and/or a visualization systemfor capturing images from the distal end of the catheter system.

The robotic interventional system 100 also includes a display system 110for displaying an image of the surgical site and interventionalinstruments 104 generated by sub-systems of the sensor system 108. Thedisplay 110 and the operator input system 106 may be oriented so theoperator can control the interventional instrument system 104 and theoperator input system 106 as if viewing the workspace in substantiallytrue presence. True presence means that the displayed tissue imageappears to an operator as if the operator was physically present at theimage location and directly viewing the tissue from the perspective ofthe image.

Alternatively or additionally, display system 110 may present images ofthe surgical site recorded and/or imaged preoperatively orintra-operatively using imaging technology such as computerizedtomography (CT), magnetic resonance imaging (MRI), fluoroscopy,thermography, ultrasound, optical coherence tomography (OCT), thermalimaging, impedance imaging, laser imaging, nanotube X-ray imaging, orthe like. The presented preoperative or intra-operative images mayinclude two-dimensional, three-dimensional, or four-dimensional(including e.g., time based or velocity based information) images andassociated image data sets for reproducing the images.

In some embodiments, the display system 110 may display a virtualvisualization image in which the actual location of the interventionalinstrument is registered (e.g., dynamically referenced) withpreoperative or concurrent images to present the surgeon with a virtualimage of the internal surgical site at the location of the tip of thesurgical instrument.

In other embodiments, the display system 110 may display a virtualvisualization image in which the actual location of the interventionalinstrument is registered with prior images (including preoperativelyrecorded images) or concurrent images to present the surgeon with avirtual image of an interventional instrument at the surgical site. Animage of a portion of the interventional instrument 104 may besuperimposed on the virtual image to assist the surgeon controlling theinterventional instrument.

The robotic interventional system 100 also includes a control system112. The control system 112 includes at least one memory and at leastone processor (not shown), and typically a plurality of processors, foreffecting control between the interventional instrument system 104, theoperator input system 106, the sensor system 108, and the display system110. The control system 112 also includes programmed instructions (e.g.,a computer-readable medium storing the instructions) to implement someor all of the methods described herein. While control system 112 isshown as a single block in the simplified schematic of FIG. 1, thesystem may comprise a number of data processing circuits with a portionof the processing optionally being performed on or adjacent the roboticassembly 102, a portion being performed at the operator input system106, and the like. Any of a wide variety of centralized or distributeddata processing architectures may be employed. Similarly, the programmedinstructions may be implemented as a number of separate programs orsubroutines, or they may be integrated into a number of other aspects ofthe robotic systems described herein. In one embodiment, control system112 supports wireless communication protocols such as Bluetooth, IrDA,HomeRF, IEEE 802.11, DECT, and Wireless Telemetry.

In some embodiments, control system 112 may include one or more servocontrollers to provide force and torque feedback from the interventionalinstrument system 104 to one or more corresponding servomotors for theoperator input system 106. The servo controller(s) may also transmitsignals instructing robotic assembly 102 to move the interventionalinstruments 104 which extend into an internal surgical site within thepatient body via openings in the body. Any suitable conventional orspecialized servo controller may be used. A servo controller may beseparate from, or integrated with, robotic assembly 102. In someembodiments, the servo controller and robotic assembly are provided aspart of a robotic arm cart positioned adjacent to the patient's body.

The control system 112 may further include a virtual visualizationsystem to provide navigation assistance to the interventionalinstruments 104. Virtual navigation using the virtual visualizationsystem is based upon reference to an acquired dataset associated withthe three dimensional structure of the anatomical passageways. Morespecifically, the virtual visualization system processes images of thesurgical site imaged using imaging technology such as computerizedtomography (CT), magnetic resonance imaging (MRI), fluoroscopy,thermography, ultrasound, optical coherence tomography (OCT), thermalimaging, impedance imaging, laser imaging, nanotube X-ray imaging, orthe like. Software is used to convert the recorded images into a twodimensional or three dimensional composite representation of a partialor an entire anatomical organ or anatomical region. An image data set isassociated with the composite representation. The compositerepresentation and the image data set describe the various locations andshapes of the passageways and their connectivity. The images used togenerate the composite representation may be recorded preoperatively orintra-operatively during a clinical procedure. In an alternativeembodiment, a virtual visualization system may use standardrepresentations (i.e., not patient specific) or hybrids of a standardrepresentation and patient specific data. The composite representationand any virtual images generated by the composite representation mayrepresent the static posture of a deformable anatomic region during oneor more phases of motion (e.g., during an inspiration/expiration cycleof a lung).

During a virtual navigation procedure, the sensor system 108 may be usedto compute an approximate location of the instrument with respect to thepatient anatomy. The location can be used to produce both macro-leveltracking images of the patient anatomy and virtual internal images ofthe patient anatomy. Various systems for using fiber optic sensors toregister and display an interventional implement together withpreoperatively recorded surgical images, such as those from a virtualvisualization system, are known. For example U.S. patent applicationSer. No. 13/107,562, filed May 13, 2011, disclosing, “Medical SystemProviding Dynamic Registration of a Model of an Anatomical Structure forImage-Guided Surgery,” which is incorporated by reference herein in itsentirety, discloses one such system.

The robotic interventional system 100 may further include optionaloperation and support systems (not shown) such as illumination systems,steering control systems, irrigation systems, and/or suction systems. Inalternative embodiments, the robotic system may include more than onerobotic assembly and/or more than one operator input system. The exactnumber of manipulator assemblies will depend on the surgical procedureand the space constraints within the operating room, among otherfactors. The operator input systems may be collocated, or they may bepositioned in separate locations. Multiple operator input systems allowmore than one operator to control one or more manipulator assemblies invarious combinations.

FIG. 2 illustrates an interventional instrument system 200 which may beused as the interventional instrument system 104 of roboticinterventional system 100. Alternatively, the interventional instrumentsystem 200 may be used for non-robotic exploratory procedures or inprocedures involving traditional manually operated interventionalinstruments, such as endoscopy.

The instrument system 200 includes a catheter system 202 coupled to aninstrument body 204. The catheter system 202 includes an elongatedflexible catheter body 216 having a proximal end 217 and a distal end ortip portion 218. In one embodiment, the flexible body 216 has anapproximately 3 mm outer diameter. Other flexible body outer diametersmay be larger or smaller. The catheter system 202 may optionally includea shape sensor 222 for determining the position, orientation, speed,pose, and/or shape of the catheter tip at distal end 218 and/or of oneor more segments 224 along the body 216. The entire length of the body216, between the distal end 218 and the proximal end 217 may beeffectively divided into the segments 224. If the instrument system 200is an interventional instrument system 104 of a robotic interventionalsystem 100, the shape sensor 222 may be a component of the sensor system108. If the instrument system 200 is manually operated or otherwise usedfor non-robotic procedures, the shape sensor 222 may be coupled to atracking system that interrogates the shape sensor and processes thereceived shape data.

The shape sensor system 222 may include an optical fiber aligned withthe flexible catheter body 216 (e.g., provided within an interiorchannel (not shown) or mounted externally). In one embodiment, theoptical fiber has a diameter of approximately 200 μm. In otherembodiments, the dimensions may be larger or smaller.

The optical fiber of the shape sensor system 222 forms a fiber opticbend sensor for determining the shape of the catheter system 202. In onealternative, optical fibers including Fiber Bragg Gratings (FBGs) areused to provide strain measurements in structures in one or moredimensions. Various systems and methods for monitoring the shape andrelative position of an optical fiber in three dimensions are describedin U.S. patent application Ser. No. 11/180,389, filed Jul. 13, 2005,disclosing “Fiber optic position and shape sensing device and methodrelating thereto;” U.S. Provisional Pat. App. No. 60/588,336, filed onJul. 16, 2004, disclosing “Fiber-optic shape and relative positionsensing;” and U.S. Pat. No. 6,389,187, filed on Jun. 17, 1998,disclosing “Optical Fibre Bend Sensor,” which are incorporated byreference herein in their entireties. In other alternatives, sensorsemploying other strain sensing techniques such as Rayleigh scattering,Raman scattering, Brillouin scattering, and Fluorescence scattering maybe suitable. In other alternative embodiments, the shape of the cathetermay be determined using other techniques. For example, if the history ofthe catheter's distal tip pose is stored for an interval of time that issmaller than the period for refreshing the navigation display or foralternating motion (e.g., inhalation and exhalation), the pose historycan be used to reconstruct the shape of the device over the interval oftime. As another example, historical pose, position, or orientation datamay be stored for a known point of an instrument along a cycle ofalternating motion, such as breathing. This stored data may be used todevelop shape information about the catheter. Alternatively, a series ofpositional sensors, such as EM sensors, positioned along the cathetercan be used for shape sensing. Alternatively, a history of data from apositional sensor, such as an EM sensor, on the instrument during aprocedure may be used to represent the shape of the instrument,particularly if an anatomical passageway is generally static.Alternatively, a wireless device with position or orientation controlledby an external magnetic field may be used for shape sensing. The historyof its position may be used to determine a shape for the navigatedpassageways.

In this embodiment, the optical fiber may include multiple cores withina single cladding. Each core may be single-mode with sufficient distanceand cladding separating the cores such that the light in each core doesnot interact significantly with the light carried in other cores. Inother embodiments, the number of cores may vary or each core may becontained in a separate optical fiber.

In some embodiments, an array of FBG's is provided within each core.Each FBG comprises a series of modulations of the core's refractiveindex so as to generate a spatial periodicity in the refraction index.The spacing may be chosen so that the partial reflections from eachindex change add coherently for a narrow band of wavelengths, andtherefore reflect only this narrow band of wavelengths while passingthrough a much broader band. During fabrication of the FBG's, themodulations are spaced by a known distance, thereby causing reflectionof a known band of wavelengths. However, when a strain is induced on thefiber core, the spacing of the modulations will change, depending on theamount of strain in the core. Alternatively, backscatter or otheroptical phenomena that vary with bending of the optical fiber can beused to determine strain within each core.

Thus, to measure strain, light is sent down the fiber, andcharacteristics of the returning light are measured. For example, FBG'sproduce a reflected wavelength that is a function of the strain on thefiber and its temperature. This FBG technology is commercially availablefrom a variety of sources, such as Smart Fibres Ltd. of Bracknell,England. Use of FBG technology in position sensors for robotic surgeryis described in U.S. Pat. No. 7,930,065, filed Jul. 20, 2006, disclosing“Robotic Surgery System Including Position Sensors Using Fiber BraggGratings,” which is incorporated by reference herein in its entirety.

When applied to a multicore fiber, bending of the optical fiber inducesstrain on the cores that can be measured by monitoring the wavelengthshifts in each core. By having two or more cores disposed off-axis inthe fiber, bending of the fiber induces different strains on each of thecores. These strains are a function of the local degree of bending ofthe fiber. For example, regions of the cores containing FBG's, iflocated at points where the fiber is bent, can thereby be used todetermine the amount of bending at those points. These data, combinedwith the known spacings of the FBG regions, can be used to reconstructthe shape of the fiber. Such a system has been described by LunaInnovations. Inc. of Blacksburg, Va.

As described, the optical fiber may be used to monitor the shape of atleast a portion of the catheter system 202. More specifically, lightpassing through the optical fiber is processed to detect the shape ofthe catheter system 202 and for utilizing that information to assist insurgical procedures. The sensor system (e.g. sensor system 108) mayinclude an interrogation system for generating and detecting the lightused for determining the shape of the catheter system 202. Thisinformation, in turn, can be used to determine other related variables,such as velocity and acceleration of the parts of an interventionalinstrument. The sensing may be limited only to the degrees of freedomthat are actuated by the robotic system, or may be applied to bothpassive (e.g., unactuated bending of the rigid members between joints)and active (e.g., actuated movement of the instrument) degrees offreedom.

The interventional instrument system may optionally include a positionsensor system 220. The position sensor system 220 may be a component ofan electromagnetic (EM) sensor system with the sensor 220 including oneor more conductive coils that may be subjected to an externallygenerated electromagnetic field. Each coil of the EM sensor system 220then produces an induced electrical signal having characteristics thatdepend on the position and orientation of the coil relative to theexternally generated electromagnetic field. In one embodiment, the EMsensor system may be configured and positioned to measure six degrees offreedom, e.g., three position coordinates X, Y, Z and three orientationangles indicating pitch, yaw, and roll of a base point or five degreesof freedom, e.g., three position coordinates X, Y, Z and two orientationangles indicating pitch and yaw of a base point. Further description ofan EM sensor system is provided in U.S. Pat. No. 6,380,732, filed Aug.11, 1999, disclosing “Six-Degree of Freedom Tracking System Having aPassive Transponder on the Object Being Tracked,” which is incorporatedby reference herein in its entirety.

The flexible catheter body 216 includes a channel sized and shaped toreceive an auxiliary tool 226. Auxiliary tools may include, for example,image capture probes, biopsy devices, laser ablation fibers, or othersurgical, diagnostic, or therapeutic tools. Auxiliary tools may includeend effectors having a single working member such as a scalpel, a blade,an optical fiber, or an electrode. Other end effectors may include apair or plurality of working members such as forceps, graspers,scissors, or clip appliers, for example. Examples of electricallyactivated end effectors include electrosurgical electrodes, transducers,sensors, and the like. In various embodiments, the auxiliary tool 226may be an image capture probe including a distal portion with astereoscopic or monoscopic camera disposed near the distal end 218 ofthe flexible catheter body 216 for capturing images (including videoimages) that are processed for display. The image capture probe mayinclude a cable coupled to the camera for transmitting the capturedimage data. Alternatively, the image capture instrument may be afiber-optic bundle, such as a fiberscope, that couples to the imagingsystem. The image capture instrument may be single or multi-spectral,for example capturing image data in the visible spectrum, or capturingimage data in the visible and infrared or ultraviolet spectrums.

The flexible catheter body 216 may also house cables, linkages, or othersteering controls (not shown) that extend between the instrument body204 and the distal end 218 to controllably bend or turn the distal end218 as shown for example by the dotted line versions of the distal end.In embodiments in which the instrument system 200 is actuated by arobotic assembly, the instrument body 204 may include drive inputs thatcouple to motorized drive elements of the robotic assembly. Inembodiments in which the instrument system 200 is manually operated, theinstrument body 204 may include gripping features, manual actuators, andother components for manually controlling the motion of the instrumentsystem. The catheter system may be steerable or, alternatively, may benon-steerable with no integrated mechanism for operator control of theinstrument bending. Also or alternatively, the flexible body 216 candefine one or more lumens through which interventional instruments canbe deployed and used at a target surgical location.

In various embodiments, the interventional instrument system 200 mayinclude a flexible bronchial instrument, such as a bronchoscope orbronchial catheter for use in examination, diagnosis, biopsy, ortreatment of a lung. The system is also suited for navigation andtreatment of other tissues, via natural or surgically created connectedpassageways, in any of a variety of anatomical systems including thecolon, the intestines, the kidneys, the brain, the heart, thecirculatory system, or the like.

While an instrument (e.g. catheter 202) is navigated through anatomicpassageways, the clinician may view an endoscopic camera image and avirtual anatomical image generated from preoperative or intraoperativeimaging. The camera image and the virtual image are registered toprovide the clinician with an intuitive navigation experience. If anendoscopic camera is not used, for example because the anatomicpassageway is not sized to accommodate a camera, registration of thedistal end of the catheter with the virtual anatomical image is neededbecause the clinician may rely on the virtual anatomical image toprovide navigation guidance and pose confidence when performing aprocedure such as a biopsy. With regions of the patient anatomy that aredynamic in normal function (e.g., heart, lungs, kidneys, liver, bloodvessels), substantially continuous dynamic registration of theinstrument and the virtual anatomical image is needed. As will bedescribed, a shape sensor located within the catheter may be used toboth model cyclic anatomical motion and track the movement of theinstrument within the moving anatomy.

FIG. 3a illustrates a general method 300 for modeling cyclic anatomicalmotion according to an embodiment of this disclosure. One or more of theprocesses 302-308 of the method 300 may be implemented, at least inpart, in the form of executable code stored on non-transient, tangible,machine-readable media that when run by one or more processors may causethe one or more processors to perform one or more of the processes302-308. At 302, a shape sensor, such as an optical fiber shape sensor,extends within an interventional instrument (e.g. catheter 202). Theinterventional instrument is inserted into a dynamic anatomicalpassageway of a patient and fixed with respect to the anatomicalpassageway. For example, the instrument may be wedged into a tightpassageway and held in place by a frictional force. Alternatively, theinstrument may be fixed using retractable anchors, inflatable fixationdevices or other anchoring mechanisms known in the art. The instrumentis constrained within the passageway but compliantly moves with thepassageways during the anatomic motion cycle. The shape sensor of theinterventional instrument is interrogated and returns strain data usedto determine a composite shape of the optical fiber, includingreconstructing the bending, twisting, compression, and other shapecharacteristics that define the overall composite shape of the opticalfiber. From this shape information, the pose of a discrete point P, suchas a distal end or other point along the instrument, may be filtered anddetermined. The shape sensor is interrogated at a plurality of timesduring a patient anatomical motion cycle, and the pose of the point Pmay be determined at each time interval. For convenience and withoutlimitation, this disclosure will describe the use of systems and methodswith reference to the respiration cycle as the anatomical motion cycle.It is understood that the same systems and methods are applicable toother forms of cyclic or otherwise predictable anatomic motion such ascardiac motion.

At 304, a set of three dimensional pose differentials is determined forthe point P with respect to at least one reference element R (See, FIG.6) at each of the plurality of time intervals. In other words, thedifference between the reference element R and the point P is determinedat each time interval within the anatomic motion cycle. The differencemay be based upon the position, such as using a position differencevector between P and R. Additionally or alternatively, the differencemay be based upon orientation. Any partial pose combination of positionand orientation differences may be used to determine differentialsbetween element R and point P. Each three-dimensional pose differentialin the set may be expressed, for example, as a three dimensional vectorbetween reference element R and the point P. The reference element R mayinclude a localization marker cable of being sensed by the previouslydescribed position sensor system (e.g., an EM sensor system). Thus, thereference element R may include electromagnetic coils or alternativelymay include infrared light emitting diodes, reflective markers, or otherelements used to determine the location of the reference element. Thereference element R may be positioned on the surface of the patientanatomy or internal to the patient anatomy at a location thatexperiences little or no movement due to the cyclical anatomical motionor to a stationary fixture external to the patient anatomy.Alternatively, the reference element R may be positioned at a locationthat experiences cyclical anatomical motion with a discernible periodicdifference between the motion of the element R and the motion of thepoint P. Suitable locations for positioning the reference element R mayinclude, for example, the sternum, the main carina (i.e., the bottom ofthe trachea where the trachea divides into the left and right mainbronchus leading to the lungs), a location on a vertebral body of thepatient's spine, or any other anatomical location that experiencesmovement with a periodic difference from the movement of the point P.

At 306, a periodic signal related to the anatomical motion cycle isdetermined from the set of three dimensional pose differentials. Forexample, where the set of three-dimensional pose differentials isexpressed as a set of three-dimensional vectors between referenceelement R and the point P, a periodic signal with well-defined phases orstages corresponding to distinct states of anatomical motion isextracted from the set of three-dimensional vectors. For example, in oneembodiment, the length of each vector is measured, and a maximum statepose differential (e.g., longest vector) and a minimum state posedifferential (e.g., shortest vector) are identified for the point P. Themaximum state pose differential is the maximum three-dimensionaldifference in position and orientation between the point P and thereference element R. The minimum state pose differential is the minimumthree-dimensional difference in the position and orientation between thepoint P and the reference element R. If, for example the anatomicalmotion is respiration, the maximum state pose differential is associatedwith the pose of point P at the distinct anatomical state of fullinspiration and the minimum state pose differential is associated withthe pose of point P at the distinct anatomical state of full expiration.The intermediate state pose differentials correspond to intermediatestages of inhalation and exhalation. From these maximum, minimum, andintermediate states, a distinct periodic signal can be extracted. Inanother embodiment, a particular aspect of the set of three-dimensionalvectors between the reference element R and the point P may be used toextract a periodic signal. For example, the periodic signal may beextracted based upon a single dimension (e.g., the x, y, orz-coordinate) differential. Alternatively, the periodic signal may beextracted based upon a differential in the angle of the vector.Alternatively, the periodic signal may be extracted based upon a changerate of displacement for the point P with respect to the referenceelement R. For example, as one or more coordinates of thethree-dimensional vector between the point P and the reference element Rchanges signs (e.g., positive to negative), maximum and minimum valuesfor the periodic signal may be extracted. Alternatively, the periodicsignal may be extracted, based at least in part upon, a mechanism (e.g.,a respirator) that is causing the anatomic motion or from a mechanism(e.g., an external respiration monitor) that is monitoring the anatomicmotion.

At 308, first and second time parameters, from the plurality of timeparameters, corresponding to first and second anatomic states,respectively, are identified and stored. For example, the first anatomicstate may be full inspiration and the second anatomic state may be fullexpiration. The use of two time parameters to correspond to two anatomicstates is illustrative and not intended to be limiting. Alternatively, asingle or three or more time parameters from the plurality of timeparameters may correspond to a single or to three or more anatomicstates, respectively. The first and second time parameters may also benormalized such that parameters t_(1st)=1 and t_(2nd)=0. These maximumand minimum time parameters may be used, for example, to model themotion cycle for an intermediate time parameter t_(x), where t_(1st)=1,. . . t_(x), . . . , t_(2nd)=0. (See, FIG. 6) In the embodiment of FIG.6, the displacements recorded at those first and second time parametersare considered the maximum pose for point P (P_(1st)) in the motioncycle and the minimum pose for point P (P_(t-2nd)) in the motion cycle,respectively. The first pose for point P (P_(t-1st)) and the second posefor point P (P_(t-2nd)) may also be stored. The process 300 may berepeated for different points on the shape sensor or when the tool hasbeen moved to a new area of interest.

FIG. 3b illustrates a general method 310 for modeling cyclic anatomicalmotion according to another embodiment of this disclosure. One or moreof the processes 312-314 of the method 310 may be implemented, at leastin part, in the form of executable code stored on non-transient,tangible, machine-readable media that when run by one or more processorsmay cause the one or more processors to perform one or more of theprocesses 312-314. At a process 312, a shape sensor, such as an opticalfiber shape sensor, extends within an interventional instrument (e.g.catheter 202). The interventional instrument is inserted into a dynamicanatomical passageway of a patient and fixed with respect to theanatomical passageway as described above for process 302. The shapesensor of the interventional instrument is interrogated and returnsstrain data used to determine a composite shape of the optical fiber,including reconstructing the bending, twisting, compression, and othershape characteristics that define the overall composite shape of theoptical fiber. From this shape information, the pose of a discrete pointP₁, such as a distal end or other point along the instrument, may befiltered and determined. The shape sensor is interrogated at a pluralityof times during a patient anatomical motion cycle, and the pose of thepoint P₁ may be determined at each time interval. Likewise, the shapesensor may be interrogated at multiple discrete points P₁-P_(n) alongthe instrument during the anatomical motion cycle to determine the poseof a set of points at each time interval.

In alternative embodiments, a shape sensor used for tracking respirationphase cycle may be mounted to the patient's chest rather than locatedinside of the patient's lungs. Alternatively, because a respirationphase cycle may be determined from the frequency of a signal withoutregard to absolute position, the frequency content of velocity andacceleration signals may also be used to determine respiration phasecycle. Accelerometers and gyroscopes, including wireless versions, usedto track velocity and/or acceleration may be used as sensors todetermine the respiration phase cycle.

A shape sensor or any other type of sensor used to track respiration maybe actuated in any of a variety of ways. For example, the sensor maybegin and/or end tracking in response to a user command such asactivation of a trigger deployable based upon motion of the user's handor foot, a verbal command, an eye gaze command, or use of usercontrolled implement such as a mouse. Sensors may also be actuated whenactuation commands to the medical instrument have terminated. Forexample, when the medical instrument reaches a target location prior toa biopsy procedure, the sensor to track respiration may be activated.Alternatively, sensors may be actuated when the user commanded positionis constant (e.g., when the user input control is maintained in aconstant position)

At 314, a periodic signal related to the anatomical motion cycle isdetermined. In one embodiment, the anatomical motion cycle may be arespiratory phase cycle. The respiratory phase cycle may be tracked asdescribed above at process 306. Other techniques for tracking therespiratory phase cycle may also be suitable. For any of the describedtechniques, the extreme phases of respiration may be differentiated bythe fact that a typical human respiration cycle lingers in theexpiration phase longer than it does in the inspiration phase.

In one example of respiration phase cycle tracking, the main carina ofthe patient anatomy may serve as the reference element R because thisreference element experiences relatively little movement duringrespiration as compared to more peripheral areas of the lung thatexperience greater displacement during inhalation and exhalation. A setof three dimensional pose differentials is determined for each pointP₁-P_(n) with respect to at least one reference element R (See, FIG. 6)at each of the plurality of time intervals. In other words, thedifference between the reference element R and the points P₁-P_(n) isdetermined at each time interval within the anatomic motion cycle. Fromthe maximum, minimum, and intermediate differential states associatedwith these points P₁-P_(n), a distinct periodic signal can be extracted.

Alternatively, various data mapping tools such as look-up tables may beused for respiratory phase cycle tracking based on the location of apoint P (e.g., the location of the point at the distal tip of thecatheter) within the lung. For example, as shown at FIG. 7, a look-uptable 600 provides a listing of locations 602 within the anatomy. In ahuman lung, for example, the locations may be associated with lobes ofthe lung (e.g., right middle lobe, left superior lobe) and/or withrespect to distance from the main carina (e.g., primary bronchi,secondary bronchi, tertiary bronchi, bronchioles). For each location inthe lung, a particular type of preferred motion measurement 604 may beapplied. For example, a Motion Measurement A for tracking a direction ofmotion (e.g., a direction perpendicular to the direction of the airway)may be associated with a Location 1 (e.g., a tertiary bronchi in theleft inferior lobe). As another example, the preferred motionmeasurement, Motion Measurement A, may be based upon the expectedprimary direction of expansion of the lungs. The preferred motionmeasurement for the lower (inferior) right lung could be along agenerally left-right axis. The preferred motion measurement for themiddle right lung could be along a generally superior-inferior axis(i.e., axis extending generally in a head-to-toe-direction). From themaximum, minimum, and intermediate differential states associated with apoint P at a particular location 602, a distinct periodic signal can beextracted.

As another example, a look-up table 610 provides a listing of locations612 within, for example, the lung. For each location in the lung, adifferent set of points along the shape sensor may be tracked to returna most accurate motion signal. For example, a Set A of shape sensorpoints (e.g., points clustered near the distal end of the catheter) maybe associated with a Location 1 (e.g., a tertiary bronchi in the leftinferior lobe). As another example, if location in the listing oflocations 612 is close to the pleura or close to rigid body structures(e.g., cartilage or bone), the expected lung movement may be small. Inthis example, if a distal end of an instrument is located close to thepleura or rigid body structures, a preferred set of shape sensor pointsmay be located along a more proximal length of the instrument, away fromthe distal end. As another example, if a location in the listing oflocations 612 is in an anatomic area that experiences rotational motioninstead of translational motion, the set of shape sensor points to betracked may be located in the area of the anatomy receiving rotationalmotion. From the maximum, minimum, and intermediate differential statesassociated with the set of points at a particular location 612, adistinct periodic signal can be extracted.

Alternatively, parallel sensors may be used to track the respirationphase cycle. For example, vision-based sensor data from a vision-basedsensor (e.g. an endoscope) positioned within the lungs that moves withthe lungs during respiration is compared to EM sensor data from an EMsensor positioned at a relatively stationary location such that theairway moves relative to the EM sensor. The difference between thetracked data from these sensors may be used to extract a distinctperiodic signal for the respiration phase cycle.

Alternatively, the respiration phase cycle may be tracked by taking arelatively noisy set of data from the shape sensor and/or other sensorsand extracting cardiac and respiration phase cycles. More specifically,a cardiac frequency (e.g., number of heart beats per minute) and arespiration frequency (e.g., number of exhalations per minute) may becomputed from the set of sensor data without regard to the phase (orstage of the cycle) for each cyclic motion. The estimated cardiac andrespiration frequencies may be used to digitally filter the sensor dataat one or more time intervals to extract phase information.

Although respiration phase cycle tracking may be performed by trackingshape sensor or other sensor signals as described above, othertechniques for tracking the respiratory phase cycle may also besuitable. For example, a respiration phase cycle may be determined byapplying a bandpass filter to the measured data (e.g., position data,velocity data, shape data) near the expected frequency of respiration(e.g., exhalations/minute) and extracting the respiration phase cyclefrom the filtered data. The bandpass filter may select only frequenciesbetween a predefined maximum and minimum frequency near the expectedfrequency of respiration.

In another example, respiration phase cycle may be determined bycomparing commanded instrument motion to sensed instrument motion. In ateleoperational surgical system, known motion commands are provided tomove the teleoperationally controlled instrument. For example, knowncommands are provided to motor actuators that control the steerabledistal end of an instrument. The actual movement may be determined fromthe shape sensor or other sensor data. The actual movement, asdetermined from the sensor data, may be subtracted from the commandedmotion to isolate the remaining motion due to anatomical motion such asrespiration.

FIG. 5 provides a three dimensional image 500 of cyclically movinganatomical passageways 502 in which an interventional instrument 504extends. The pose of the instrument 504 is measured at point P for aplurality of time intervals during the cyclic anatomical motion, and thepose is compared to the reference element R for each of the timeintervals. The maximum pose differential occurs at P_(t-1st) and theminimum pose differential occurs at P_(t-2nd).

FIG. 4 illustrates a method 350 of tracking an interventional instrument(e.g. catheter 202) within a patient anatomy during cyclic anatomicalmotion after modeling the cyclical anatomical motion using method 300,310 or another suitable method. One or more of the processes 352-370 ofthe method 350 may be implemented, at least in part, in the form ofexecutable code stored on non-transient, tangible, machine-readablemedia that when run by one or more processors may cause the one or moreprocessors to perform one or more of the processes 352-370. At 352,anatomical image data from a preoperative or intraoperative anatomicalimage for first state of cyclical movement is received for processing.For example, if the anatomic cycle is respiration, a patient may beimaged at full inspiration, the state that corresponds to the first ormaximum state of the cyclical movement. At 354, anatomical image datafrom a preoperative or intraoperative anatomical image for a secondstate of cyclical movement is received for processing. For example, apatient may be imaged at full expiration, the state that corresponds tothe second or minimum state of the cyclical movement. In alternativeembodiments, the first and second states of cyclical motion need notcorrespond to maximum or minimum extrema, but may correspond to otheridentified stages of the cycle. In still other alternative embodiments,image data from a single state of cyclical motion or more than twostates of the cyclical motion may be matched to the extracted periodicsignal.

At 356, during the interventional procedure, the shape sensor of theinterventional instrument is interrogated at a time interval t=t_(x),and a pose dataset for a plurality of points along the shape sensor att=t_(x) is collected. To register the interventional instrument with apreoperative or intraoperative image at t=t_(x), a coarse matchingprocedure, optionally supplemented by a fine matching procedure, may beused.

To perform a coarse matching, at 358 a three-dimensional differencebetween the pose dataset of the plurality of points along the shapesensor at t_(x) and the anatomical image data at the first state ofcyclical motion is calculated to determine a pose match score for thefirst state. Additionally, a three dimensional difference between thepose dataset of the plurality of points along the shape sensor at t_(x)and the anatomical image data at the second state of cyclical motion iscalculated to determine a pose match score for the second state. Thepose match scores for the first and second states indicate which statehas anatomical image data most similar to the shape sensor at t_(x). Thestate with the best pose match score, i.e. the state that most closelymatches the shape sensor, is identified as the matching anatomical imagedata set. The best pose match score may be based, for example, upon oneor more of a comparison of distance, orientation, shape matches, ornavigation decision history.

Optionally, at 370, a virtual image of the interventional instrument isdisplayed in combination with the matched anatomical image. For example,the image of the interventional instrument may be gated and displayedwith the matched anatomical image from either the first or second state.The displayed instrument shape may be modified to align with the matchedanatomical image. Alternatively, the image may be modified to fit withthe measured shape. Also optionally, the matched anatomical image may bepresented simultaneous (e.g., in a different window of a display or on adifferent display) with a current endoscopic image from theinterventional instrument.

FIG. 6 illustrates an interventional instrument pose dataset P1-P6compared to the anatomical image data at first state P1 _(1st)-P6 _(1st)and second state P1 _(2nd)-P6 _(2nd) for generating a pose match score.The points P1 _(1st)-P6 _(1st) may follow, for example, a centerlinethrough the lumen of the anatomic passageway 400 in the state of fullinhalation. The points P1 _(2nd)-P6 _(2nd) may follow, for example, acenterline through the lumen of the anatomic passageway 400 in the stateof full exhalation. A three dimensional difference ΔP1 _(1st) isdetermined between the instrument point P1 and the point P1 _(1st) fromthe anatomic image data from the state of full inhalation. A threedimensional difference is further computed for each instrument point anda corresponding point from the anatomic image at full inhalation. Theinstrument insertion length may be used, when the instrument is moving,to approximate the association between the instrument points P1-P6 andthe anatomic image points P1 _(1st)-P6 _(1st). Instrument and imagepoints can be associated using best-fit techniques. For example, for agiven instrument point, the closest image point that is within ananatomic passageway may be chosen. The instrument insertion length maybe used to limit the choice to points in the image that areapproximately the insertion length away from the insertion point. Thethree-dimensional differences at each point are combined (e.g., added oraveraged) to generate the pose match score for the first state.Likewise, a three dimensional difference ΔP1 _(2nd) is determinedbetween the instrument point P1 and the point P1 _(2nd) from theanatomic image data from the state of full exhalation. A threedimensional difference is further computed for each instrument point anda corresponding point from the anatomic image at full exhalation. Thethree-dimensional differences at each point are combined (e.g., added oraveraged) to generate the pose match score for the second state.

To enable continuous tracking and dynamic registration of theinterventional instrument with the endoscopic image and/or the virtualanatomical image and to avoid the jerky display associated with thecoarse matching and gating techniques, a fine matching procedure may beperformed. Referring again to FIG. 4, to perform fine matching, at 360lumen models are extracted from the image data at the first and secondstates of cyclical movement. The lumen model may be any form of anatomicrepresentation including, for example, a centerline model, animage-voxel model, a geometric (e.g., mesh) model, or a parametricmodel. On example of a method of modeling a branching anatomy isprovided in U.S. patent application Ser. Nos. 13/893,040 and 13/892,871filed May 13, 2013, disclosing “Systems and Methods for Registration ofa Medical Device Using a Reduced Search Space,” which is incorporated byreference herein in its entirety. Another example of a method ofmodeling a branching anatomy using a mesh deformation technique isprovided in U.S. Provisional Pat App. No. 61/935,547 filed Feb. 4, 2014,disclosing “Systems and Methods for Non-rigid Deformation of Tissue forVirtual Navigation of Interventional Tools,” which is incorporated byreference herein in its entirety.

At 362, the determined first and second time parameters (t_(1st)=1,t_(2nd)=0) from process 308 are associated with the respective firststate and second state lumen models. The first and second timeparameters are determined from, for example, the modeled anatomic motionas described in method 300 or other cyclic motion modeling techniques.At 364, a plurality of intermediate lumen models are created byinterpolating intermediate movement states between the first and secondstates for a plurality of time parameters (t_(1st)=1, . . . t_(x), . . ., t_(2nd)=0) in the modeled motion cycle.

At 366, a pose match score is calculated for each intermediate lumenmodel by performing a three dimensional comparison between theinterventional instrument pose dataset P1-P6 and corresponding points oneach intermediate lumen model. A pose match score is also calculated forthe lumen models at the maximum and minimum states. At 368, the lumenmodel with the best pose match score, i.e. the lumen model that mostclosely matches the shape of the shape sensor, is identified as thematched lumen model. At 369, a matched anatomical image data set isgenerated from the matched lumen model. Generating the matchedanatomical image data set may include applying a deformation vectorfield. The use of a deformation vector field to generate image data isdescribed in greater detail in U.S. Pat App. No. 61/935,547, which isincorporated by reference herein in its entirety. At 370, optionally, avirtual image of the interventional instrument in the shape at t_(x) isdisplayed in combination with the matched anatomical image data. Alsooptionally, the matched anatomical image may be presented simultaneous(e.g., in a different window of a display or on a different display)with a current endoscopic image from the interventional instrument.

FIG. 9 illustrates a method 700 of using the periodic signal data from amethod 300 or 310 to improve an interventional procedure using aninterventional instrument (e.g. catheter 202) within a patient anatomy.One or more of the alternative processes 702-722 of the method 700 maybe implemented, at least in part, in the form of executable code storedon non-transient, tangible, machine-readable media that when run by oneor more processors may cause the one or more processors to perform oneor more of the processes 702-722. Optionally, at a process 702anatomical image data from a preoperative or intraoperative anatomicalimage for first state of cyclical movement is received for processing.For example, if the anatomic cycle is respiration, a patient may beimaged at full inspiration, the state that corresponds to the first ormaximum state of the cyclical movement. Optionally, at a process 704anatomical image data from a preoperative or intraoperative anatomicalimage for a second state of cyclical movement is received forprocessing. At a process 706, periodic signal data related to ananatomical motion cycle (e.g., the periodic signal related to therespiratory phase cycle as determined in methods 300 or 310) isreceived. At a process 708, the received image and signal data may beprocessed by a control system (e.g. control system 112) to improveaspects of planning, navigation, and instrument command during aninterventional procedure.

Optionally, at a process 710, the guidance information displayed to aclinician on a display (e.g., display 110) is modified using therespiratory phase cycle data. For example, the location of a targetanatomical structure (such as a tumor to be biopsied) may be adjusted inthe displayed image based upon the current stage of the respiratoryphase cycle. Adjusting the displayed image may include interpolating theimage data from the first and/or second state of cyclical movement tothe current stage of the respiratory phase cycle. The guidanceinformation may also be modified by deforming the images, anatomicmodel, or other anatomic representation based on a best match image oron an interpolated representation, for example as described in method350.

Another technique for modifying displayed guidance information isstabilizing a camera view (e.g. an endoscope view) to give theappearance of a stationary anatomy to the clinician viewing the display.For example, the operation of the camera may be synchronized with therespiratory phase cycle data so that a zoom operation is timed to therespiration. The zoom effect may be accomplished, for example, byphysically moving the camera along the axis of the anatomic passageway,by operating a digital zoom feature of the camera or an image processor,or by operating an optical zoom feature of the camera.

Another technique for modifying the displayed guidance information isdisplaying an indicia of uncertainty together with the image of theanatomy (e.g. single image, model, other anatomic representation) andthe interventional instrument in the sensed position and/or shape. Theanatomic representation may be a matched image/model from the first orsecond state of anatomic motion or may be an interpolated image/model.The indicia of uncertainty may be the clarity of the instrument image,such as a blurry image that varies as the phase of respiration variesfrom the phase at which image was acquired. Other indicia of uncertaintyinclude a graphical bubble, color indicator, or alphanumeric indicatorthat vary as the phase of respiration varies from the phase ofrespiration at which the image of the anatomy was acquired.

At a process 712, improved filtering and/or navigation accuracy may beachieved using the respiratory phase cycle data. For example, thematched or interpolated airway model, (generated as previouslydescribed) may be used, together with the measured shape of theinterventional instrument to predict the airway in which the distal endof the interventional instrument is positioned. This prediction may bebased upon which airway in the matched or interpolated airway model bestfits the shape of the interventional instrument. In another example, therespiratory phase information may be used to perform the transformation(e.g., translation, rotation, and sizing) of the anatomic model to apatient frame of reference. Transformation methods such as iterativeclosest point (ICP)-based algorithms, vision-based tracking, or externaloptical-based tracking may adjusted by removing the open-looprespiratory motion, as described by the respiratory phase cycle data,from the sensor data used to estimate the transformation.

Another technique for improving temporal filtering and reducing lag timeinvolves filtering the data from the EM coil sensors based upon theknown respiration phase. For example, the estimated respiration motionmodel may be used to anticipate when the EM sensor data will register aquick movement (e.g. a quick motion associated with sharp inhalation).During this quick motion phase of the respiration cycle, the sensor datamay be filtered based upon (including averaged or compared to) anexpected signal or sensor measurement.

Another technique for improved sensor data filtering includes using therespiratory and/or cardiac phase cycle data to filter respiration-basedmotion and cardiac-based motion from the measured shape sensor signal.With the anatomic motion removed from the sensor signal, the displayedshape of the interventional instrument may be stabilized.

Another technique for improving navigation includes using therespiratory phase cycle data to assist in lumen navigation selection.Decisions regarding the next lumen to which the medical instrumentshould be navigated generally have a higher level of confidence whendetermined at the same phase of respiration as the phase at which theimage was captured. Without respiratory phase cycle data, lumenselection navigation decisions made at time instances between the phasesat which the images were captured have a lower level of confidence.Because the respiratory phase cycle data provides information about thelocation of the moving lumens at various times during the respirationcycle, lumen selection navigation decisions may be made with greaterconfidence.

At a process 714, the respiratory phase cycle data may be used to adjustthe airway motion model. For example, a single recorded patient-specificimage (e.g., a CT image) may be animated or modified based upon ageneric (e.g., non-patient specific) anatomic model or anatomic atlas(e.g., a two- or three-dimensional view of the airway tree) for themeasured stage of the respiratory phase cycle. Another technique foradjusting the airway motion model uses the respiratory phase cycle datato scale the amplitude of the displayed or used deformation. Morespecifically, if a patient's normal breathing pattern does not extendall the way to full inhalation and/or full exhalation (the extremes asoriginally imaged), the actual measured deformation, as determined fromthe respiratory phase cycle data, is used to adjust the displayed motionof the anatomic model to the patient's normal breathing pattern.

At a process 716, the commands issued by the user, via the controlsystem, to the motors actuating the medical instrument may be adjustedbased upon the respiratory phase cycle data. For example, a periodicoffset may be added to the command signals to move the distal tip of theinterventional instrument with the movement of the anatomy. The periodicoffset may be determined from the respiratory phase cycle data tominimize or eliminate the relative movement of the distal tip of theinterventional instrument relative to the anatomy.

Another technique for adjusting the instrument commands includesproviding the user with a timing indicator which identifies to the userthe optimal time(s) to command movement of the interventionalinstrument. For example, based upon the respiratory phase cycle data, atiming indicator may be provided when the airways experience the leastamount of movement so that a clinician may intubate a patient at thetime in the respiratory cycle when the risk of injury, due to anatomicmovement, is lowest. Alternatively, the control system 112 may preventcommanded motion of the instrument at stages of greatest movement.Alternatively, control system 112 may delay the commanded motion until astage of minimal respiratory movement is reached, as predicted by therespiratory phase cycle data. Alternatively, the trajectory of thecommanded motion may be synchronized with the anatomic motion cycle.

Another technique for adjusting the instrument commands includesadjusting a commanded position or orientation of the interventionalinstrument to minimize the amount of bending in an auxiliary tool (e.g.a biopsy needle) that extends from the catheter. More specifically thecommanded approach angle or approach position for the needle may bemodified to minimize the degree to which the needle pivots about orbends as it passes through the airway wall.

At a process 718, the actual respiration motion (including particularphases of the motion such as the actual inspiration and expirationextrema) may be monitored and compared to the motion expected based uponthe respiratory phase cycle data. If a predefined difference between theexpected motion and the actual motion is measured, the clinician may beprovided with a visual, audible, or tactile alert of a potentiallydangerous situation. Alternatively, if a predefined difference betweenthe expected motion and the actual motion is measured, the respiratoryphase cycle data and the resulting expected motion model may be adjustedfor the particular region of the lung in which the instrument islocated. The adjustment may occur responsive to a user command allowingthe adjustment or may be made without user input.

At a process 720, the respiratory phase cycle data may be used to matchmodeled and intraoperative images or models. For example, the anatomicmodel (e.g. a CT-based model) at a particular stage of respiration maybe matched and displayed with the intraoperative camera image (e.g. theendoscope image) only at the same respiration phase (or period aroundthe same respiration phase) as the anatomic model. Likewise, theanatomic model at a particular stage of respiration may be matched anddisplayed with intraoperative fluoroscopic images only at the samerespiration phase (or period around the same respiration phase) as theanatomic model.

At a process 722, the respiratory phase cycle data may be used todetermine a robust parking location (i.e., location from which anauxiliary tool is deployed from the catheter) for the catheter fromwhich the biopsy needle (or other auxiliary instrument) may be extendedto conduct an interventional procedure such as a biopsy. The knownmotion of the airways may indicate (to a clinician or the controlsystem) particular airways or locations within airways that may be moresuitable than others for parking the catheter to perform the proceduresuch that the catheter does not become unexpectedly dislodged from itsparked location due to the respiration cycle. Identifying a suitableparking location may be based upon the respiratory phase cycle data, butalso may be determined from an anatomical atlas of the passageways orfrom the full inspiration/full expiration preoperative images.

FIG. 10 illustrates a method 750 for adjusting an anatomic model or ananatomic image based upon intraoperative imaging. One or more of theprocesses 752-758 of the method 750 may be implemented, at least inpart, in the form of executable code stored on non-transient, tangible,machine-readable media that when run by one or more processors may causethe one or more processors to perform one or more of the processes752-758. At a process 702 anatomical image data from a preoperativeanatomical image for first state of cyclical movement is received forprocessing. For example, if the anatomic cycle is respiration, a patientmay be imaged at full inspiration, the state that corresponds to thefirst or maximum state of the cyclical movement. At a process 754anatomical image data from a preoperative anatomical image for a secondstate of cyclical movement is received for processing. At a process 756,intraoperative images (e.g. fluoroscopic images) are obtained for theanatomy at the first state of cyclical movement (e.g., full inspiration)and at the second state of cyclical motion (e.g., full expiration). Aset of images may also be obtained for states of cyclical movementbetween the first and second states. At a process 758, physical featuresmay be identified in the intraoperative images that correspond tofeatures in the preoperative images. Extracting a sufficient quantityand quality of features that correspond between the preoperative andintraoperative images allows the preoperative images to be adjusted insize, scale, orientation, and/or position based upon the intraoperativeimages. For example, the top-to-bottom size of the images for thepreoperative state may be measured and scaled to the intraoperativeimages so that intermediate states of the preoperative images may begenerated that follow and depict the motion of the intraoperativeintermediate images.

One or more elements in embodiments of the invention may be implementedin software to execute on a processor of a computer system such ascontrol system 112. When implemented in software, the elements of theembodiments of the invention are essentially the code segments toperform the necessary tasks. The program or code segments can be storedin a processor readable storage medium or device that may have beendownloaded by way of a computer data signal embodied in a carrier waveover a transmission medium or a communication link. The processorreadable storage device may include any medium that can storeinformation including an optical medium, semiconductor medium, andmagnetic medium. Processor readable storage device examples include anelectronic circuit; a semiconductor device, a semiconductor memorydevice, a read only memory (ROM), a flash memory, an erasableprogrammable read only memory (EPROM); a floppy diskette, a CD-ROM, anoptical disk, a hard disk, or other storage device, The code segmentsmay be downloaded via computer networks such as the Internet, intranet,etc.

Note that the processes and displays presented may not inherently berelated to any particular computer or other apparatus. The requiredstructure for a variety of these systems will appear as elements in theclaims. In addition, the embodiments of the invention are not describedwith reference to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the invention as described herein.

While certain exemplary embodiments of the invention have been describedand shown in the accompanying drawings, it is to be understood that suchembodiments are merely illustrative of and not restrictive on the broadinvention, and that the embodiments of the invention not be limited tothe specific constructions and arrangements shown and described, sincevarious other modifications may occur to those ordinarily skilled in theart.

What is claimed is:
 1. A method comprising: receiving a first set ofimage data representing a plurality of passageways at a first cyclicalmotion state; receiving a second set of image data representing theplurality of passageways at a second cyclical motion state; receivingpose data for a plurality of points describing a shape of aninterventional instrument positioned within the plurality of passagewaysat a first time parameter; comparing the shape of the interventionalinstrument at the first time parameter from the pose data to the firstset of image data and the second set of image data, wherein thecomparing includes: assigning a first match score to the first set ofimage data and a second match score to the second set of image data bycomparing each of the first set of image data and the second set ofimage data to the shape of the interventional instrument at the firsttime parameter; and determining, from among the first set of image dataand the second set of image data, a selected set of image data that mostclosely matches the shape of the interventional instrument at the firsttime parameter based on the first match score and the second matchscore; based on the comparing, identifying a phase of a cyclicalanatomical motion that occurred at the first time parameter; generatinga command signal indicating an intended movement of the interventionalinstrument relative to the plurality of passageways, the command signalincluding an instruction for operation of at least one actuatorcontrolling movement of the interventional instrument; adjusting, by acontrol system, the command signal to include an instruction for acyclical instrument motion of the interventional instrument based on thephase of the cyclical anatomical motion that occurred at the first timeparameter; and causing the intended movement of the interventionalinstrument relative to the plurality of passageways based on theadjusted command signal to compensate for the cyclical anatomicalmotion.
 2. The method of claim 1 further comprising: determining, fromthe first set of image data, a first model of the plurality of thepassageways at the first cyclical motion state; determining, from thesecond set of image data, a second model of the plurality of thepassageways at the second cyclical motion state; and determining, byinterpolation, an intermediate model of the plurality of the passagewaysat an intermediate cyclical motion state between the first cyclicalmotion state and the second cyclical motion state that corresponds tothe shape of the interventional instrument.
 3. The method of claim 2further comprising: generating a three dimensional image of theplurality of passageways based on the intermediate model and displayingan image of the interventional instrument in combination with the threedimensional image of the plurality of passageways.
 4. The method ofclaim 3 wherein generating a three dimensional image of the plurality ofpassageways from the intermediate model includes: generating adeformation vector field from the first model of the plurality ofpassageways and the second model of the plurality of passageways at thefirst and second cyclical motion states; and deforming one of the firstset of image data and the second set of image data that most closelymatches the shape of the interventional instrument at the first timeparameter using the deformation vector field.
 5. The method of claim 2wherein determining the intermediate model comprises: from the firstmodel and the second model, interpolating a plurality of models of theplurality of the passageways at intermediate cyclical motion statesbetween the first and second cyclical motion states; comparing the shapeof the interventional instrument at the first time parameter to theplurality of models at the intermediate cyclical motion states;assigning a match score to each of the plurality of models; andselecting the intermediate model from the plurality of models based onthe respective match score indicating that the selected intermediatemodel most closely matches the shape of the interventional instrument atthe first time parameter.
 6. The method of claim 1 further comprising:receiving, within the pose data, a first subset of the pose data for anidentified point on the interventional instrument fixed within and incompliant motion with the plurality of passageways for a plurality oftime parameters; determining, from the first subset of the pose data, apose differential for the identified point with respect to an anatomicalreference point at each of the plurality of time parameters; identifyinga first state pose differential and a second time parameter from theplurality of time parameters; identifying a second state posedifferential and a third time parameter from the plurality of timeparameters; and modifying one of the first set of image data and thesecond set of image data that most closely matches the shape of theinterventional instrument at the first time parameter to generate anintermediate set of image data corresponding to a fourth time parameter,wherein the fourth time parameter is between the second and third timeparameters.
 7. The method of claim 6 further comprising: displaying animage of the interventional instrument in combination with anintermediate model of the plurality of the passageways at anintermediate cyclical motion state between the first cyclical motionstate and the second cyclical motion state that corresponds to the shapeof the interventional instrument.
 8. The method of claim 6 whereindetermining the pose differential for the identified point includesdetermining a displacement of the identified point with respect to theanatomical reference point at each of the plurality of time parameters.9. The method of claim 6 wherein determining the pose differential forthe identified point includes determining a change rate of displacementfor the identified point with respect to the anatomical reference pointat each of the plurality of time parameters.
 10. The method of claim 9wherein identifying the first state pose differential and the secondstate pose differential for the identified point includes determining asign change for the change rate of displacement between two timeparameters in the plurality of time parameters.
 11. A method comprising:receiving a first anatomic model representing an anatomic region at afirst point in an anatomical cyclical motion; receiving a secondanatomic model representing the anatomic region at a second point in theanatomical cyclical motion; receiving pose data representing a shape ofan interventional instrument at a point in time, wherein theinterventional instrument is disposed within the anatomic region at thepoint in time; determining a phase of the anatomical cyclical motionthat occurred during the point in time by: comparing the first anatomicmodel to the shape represented by the pose data; and comparing thesecond anatomic model to the shape represented by the pose data; andgenerating a command signal indicating an intended movement of theinterventional instrument relative to the anatomic region, the commandsignal including an instruction for operation of at least one actuatorcontrolling movement of the interventional instrument; adjusting, by acontrol system, the command signal to include an instruction for acyclical instrument motion of the interventional instrument based on thephase of the cyclical anatomical motion; and causing the intendedmovement of the interventional instrument relative to the anatomicregion based on the adjusted command signal to compensate for thecyclical anatomical motion.
 12. The method of claim 11, wherein thedetermining of the phase of the anatomical cyclical motion that occurredduring the point in time further includes: interpolating a set ofintermediate anatomic models representing the anatomic region atrespective points in the anatomical cyclical motion that areintermediate to the first point and the second point; comparing the setof intermediate anatomic models to the shape represented by the posedata to determine corresponding scores; and based on the comparing,selecting an anatomic model that most closely matches the shaperepresented by the pose data from among: the first anatomic model, thesecond anatomic model, and the set of intermediate anatomic models. 13.The method of claim 11, wherein the first point in the anatomicalcyclical motion and the second point in the anatomical cyclical motionrepresent extrema of the anatomical cyclical motion.
 14. The method ofclaim 11 further comprising, based on the determined phase of theanatomical cyclical motion, providing an indicator during a period ofreduced movement in the anatomical cyclical motion.
 15. The method ofclaim 11 further comprising: based on the determined phase of theanatomical cyclical motion, determining a difference between a measuredmotion and an expected motion during the anatomical cyclical motion; andproviding an indication of the difference between the measured motionand the expected motion.
 16. The method of claim 15 further comprising:modifying the expected motion based on the difference between themeasured motion and the expected motion.
 17. The method of claim 11,wherein the receiving of the first anatomic model includes receiving afirst set of image data representing passageways of the anatomic regionat the first point in the anatomical cyclical motion and generating thefirst anatomic model from the first set of image data; and wherein thereceiving of the first anatomic model includes receiving a second set ofimage data representing the passageways of the anatomic region at thesecond point in the anatomical cyclical motion and generating the secondanatomic model from the second set of image data.
 18. The method ofclaim 11, further comprising: determining a first score from thecomparison of the first anatomic model to the shape represented by thepose data; and determining a second score from the comparison of thesecond anatomic model to the shape represented by the pose data.
 19. Themethod of claim 18, wherein the determining of the phase of theanatomical cyclical motion that occurred during the point in timefurther includes: generating a third anatomic model representing theanatomic region at a third point in the anatomical cyclical motion thatis intermediate to the first point and the second point; comparing thethird anatomic model to the shape represented by the pose data todetermine a third score; and based on the first score, second score, andthird score, selecting an anatomic model that most closely matches theshape represented by the pose data from among: the first anatomic model,the second anatomic model, and the third anatomic model.
 20. The methodof claim 19, wherein the generating of the third anatomic model includesinterpolating a state of the anatomic region based on the first anatomicmodel and the second anatomic model.